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ABSTRACT 

Guidelines and criteria for constructing 
to evaluate early childhood special education programs 
The information may be used to identify parent 
attitudes and staff attitudes and needs. Attention is directed to the 
focus,, title, purpose statement, directions, demographic questions, 
set of items and f responses, and structure and format^ Determining the 
focus of the questionnaire before its- construct ion involves 
identifying the following: the purpose of the questionnaire, the 
persons who will use the results > and the specific group that will 
answer the questionnaire. In addition to a title that is concise 
understandable, and reflects the content of, the instrument, an ' 
introductory statement is needed that specifies the purpose, how the 
data will be used, anonymity, and motivators for respondents to 
participate. Directions for completing the instrument need to be* 
written at the language level tof the respondents and should indicate 
the approximate amount of time required to complete the s 
questionnaire. The demographic section should request information 
limited to those variables that will be used in the data analysis, 
assist in the actual writing of questionnaires, 15 



To 



produces 
on the 



... _ , suggestions are 

presented for developing rating scales, along with tips on writing 
qualitative items (e.g., items which elicit information on job 
classification, sex); and supply items (i.e., the respondent 
the response .rather than selecting an alternative provided 
questionnaire). A sample questionnaire is included. (SEW) 
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CONSTRUCTING QUESTIONNAIRES 
FOR EARLY CHILDHOOD- SPECIAL EDUCATION -PROGRAMS 

if ■ ' ' 

' The purpose of this TADScript is to provide the reader with a set of 

guidelines and\criteria for obstructing a questionnaire. Questionnaires are 

needed for collecting data aj^eltciting responses from people about their ( 

feelings, needs, attitudes, and behaviors toward themselves- or other objects. 

As unique infonnation-gat|^ing/devices^ questionnaires hold certain advan- 

/ i 

tages: 

• A wide, large of persons can be surveyed. for low cost and time. 

• Most questionnaires afe completed anonymously, so answers are honest 
and objective. 

• The respondent? #ay £ake as much time as they wish to fill out the 
questionnaire arfd tnereby giive well thought-out answers. 

.tfi 

• Results are uniform because all persons respond to the same questions. 

/ " 

• This uniformity usually provides results which are easy to summarize 
and interpret. 

Let's consider the disadvantages: , /• *• 

• Most often, some questionnaires are not returned. Consequently* the 
quantity and quality of information diminishes. As the percentage of fi 
non-returns increases, the accuracy of the results detfeases. 

• The validity of the response on a ^questionnaire is often uncertain. We 
know nothing of the attitude of th£ respondents when they fill out a 
questionnaire and therefore do not know if their responses are colored 
by extreme interest, complacency or hostility. We also cannot be cer- 
tain that the person who received the questionnaire was the person who 

• completed the form. 3 • ^ ■ ; * V^T 

• Another major disadvantage is misinterpretation of the questron, lead- 
ing to an inaccurate response. jThis problem can be a result of inade- 
quately worded questions or respondents who are not capable of 'answer- 
ing the 'questions. ' ^ 

• A questionnaire's uniformity does not permit the flexibility to explore 
and gather additional information methods \SLUCh as 'interviews. 
The amount and quality of the information recei v|stf must be based solely 
on the questions asked. 



In spite of these disadvantages, the questionnaire remains a powerful and 
useful tool for gathering information. You-may use it to identify staff 
development needs, to obtain feedback after workshops and demonstration 
visits, or to learn about parental opinion of the quality of children's 
programs. Please remember, however, that, the power and usefulness of this' 
tool are related. directly to the quajity of care you give to the 
questionnaire's corvtruction. 
v . In the remainder of the TADScript we will provide you with guidelines and 
criteria for constructing questionnaires, followed by explanations and 
examples of good and blad practices.. The seven sections of this paper * 

• v • ^ ( 

represent areas you should consider carefully as ycJu design your 
questionnaire. The sections: - 

I. Focus 1 '* ' ' - 

II. Title ' 
I Hi Purpose Statement^ . „ • 

IV. Directions 
, ^V. Demographic Section 
VI. Set of Items and Responses 
v ^ Structure and format * 

; To enhance learning, we advise you to review Figure 1, a questionnaire 
. desfgned to'gather information frorn the parents of children participating in a 
Handicapped Children's Early Education ^Program (HCEEP) project* (Musumeci % 
Koen, 1980). Portions^f the instrument are referred to 1 throughout the paper. 

Later, as you read the discussi6hs presented'in the paper, your understanding* 

.> 

of the writing process should enable you to complete your own-questionnaire. 
To make sure your completed questionnaire is adequate, compare it to 



information presented in Figure 6 (page 56), a list of guidelines and criteria . 
summaFtzed from the text of-thfs book. # °- 

ERIC ' v ^5 ^ 



Figure 1 



PRESCHOOL HANDICAPPED PROGRAM 
PARENT QUESTIONNAIRE" 



Czntzn Vox RfeAou/tce Managmznt, Inc. tt> conducting an evaluation oi thz BOCES VazAchool Handicapped Pkognam, Thz bat>ic puxpotz ojj thz 
evaluation <ti> to pnovidz a deAcAiption oi how thz pKognam it> functioning- -ltd accomplUhmzntfy, contt/iaintb and concern. 

Vou/C cjogpeAation and pa/ttictpatLon with tkU zvaljiation zbloKt wttl givz ul thz opportunity- to obtain valuable. infonstutAjon fnom p&u>on6 
involved with mognam &znviczA. / f * 

Thz attached i Mi&t/iumznt m dtiignzd to document you/i Kzaction* to and pznczption* ojj thz ?KzAchool Vnogtuw. Ml Kz&ponteA wilt bz hzld 
in tttiiet con&idzncz, and only lutmvuzzd data. wttt bz jfozbzntzd in tub&zquznt Kzponte*. VoutC tignatu/iz i£> optional, Ait plo&Az supply thz 
lollowing^ in£o/umtion which will bz uAzd ioK composition JpuxpotzA . • 



. 1. When was your child first enrolled In the Preschool Program? 
2. In which program was your child first ^^plled? (Check One) 



3. In whkh program Is your child presently enrolled? (Check One) 



4. If your child Is In the classrocyn program , please supply the following Information: 

a) Name of teacher: 



^ 1 Session child, attends: 

5. Were you ever a parent volunteer In the cla^room program? (Check One) 
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(Month) > (Year) 

Classroom Program 4 

_____ Home Program, 

Cl-assroom Program 

Home Program 



(Morning) 





(Afternoon) 



No 



4 



\ 



l: F TOm what source did you first learn 'abo\ the" Preschool Handicapped Program? (Check One, V , • 



Friend/Other PSHP Parent 
* Nursery School . 
Newspaper/Radio 
Social Service Agency 



/hyslcian 

Posters/Flyers 

Other 



r 



(specify) 



Child Assessment Conferences 
CI assroom Visl ts/Observatlons 
Screenings 

Parent group Meetings 
^ s (Number attended: ) 



IEP Conferences 

0 

Other Conferences 
Classroom Volunteering 
Volunteer training Program 
Other 



(specify) 



t 

3. How effective were these parent meetings/activities In; 



8 



a. helping you understand the Preschool Handicapped Program 



b, helping you understand your child's handicap 



LEVEL OF EFFECTIVENESS 



Very 



c improving your attitudes about .your child's handicap 



d. increasing your skills in working with your child * 



ERIC 



() e. giving you moral support as a parent 



Fairly 



Somewhat 
# — 



Hardly 



Not At All 



SECTION II;' Reactions to Program 



A, Knowledge 




/ , 



' Low 



LEVEL OF UNDERSTANDING 



high 



* 1* 

1 » 


V 

Ph41ncnnhv/ nf f ho hw\ni*3m 

rH i i ubupny u r tne p ruy/aiii w 


J 




a 

. f . 


3 


4 


5 


? 


— 5 * — ; — ~- 

uua i b/ purpubcjy ut Lfic prugraui 




v . 






4 


r 

5 


J. - 


— — '-■ — t- 

jut pen ing/ p i aceuien t proceaures 1 


i 






3 


* 4 


5 


A 

H . 


i ransaisc 1 p 1 1 nary tearm assessment 




> 


9 * 


3 


4 


5 


c 

D . 


jedemng nieLnoas ut program* 










4 


r 

. 5 


6. 


Methods for teaching chlfd at home 


v 


■ 


•2 


3 

*■ 


4 


5 


7. 


Methods for managing child's behavior 


1 




« 2 * 


V 3 


4 


5 


8. 


* ! : J ■ s * — 

Legal rights as parents 

; 






2 


3 


4 


*> 


9. 


Educational rights of child 

— * & * 




— 


2 


3 


4 . 


* 5 


lO- 


Handicapping condition of child. 






• 2 


1 1— 

3 K ' 


4 


• 5 ' 


ll. 


Services avatflablfc In -community ^ 






- 2 


3 


4 


5 




/ 

i ■ 

r 




<* 


4 ' 


♦ 




<3 



B. « Atti tudes 

n > ■ 



tflnzvUoMi Plzuz indit*** unnn **o» 0 * n$ tnttiiattlm 
5 lhigk)^o/L zach o& thz toUowing itzmt>. 


<mk thz Ptiuohool Handicappzd Vfuognm by cXAdUng gnz numbvi fam 1 

if ; • * 


llow) to 


• 

. . l. o Preschool Handicapped Program In general 


* * 

Low 


LEVE4_ OF SATIS FAPTTflN 

. , n 




High 


1 2 


3 


4 


c 


2. Instructional methods used 


1 2 


3 


4 


c 
0 


3. Effectiveness of staff 


1 2 


3 


4 


c 
0 


4. Frequency of contact with teachers 

a 


: V 

\ - i V . 2 


3 


4 


c 
0 


5. Materials used 


1 2 


3 . 


4 


c 
0 


6. Your Involvement with ^he program 


1 2 




A 

H 


c 
3 


* 7. Opportunities for your suggestions 


1 2 


3 




c 
D 


8. —Screening/placement procedures 

\ — - — * — 


1 2 


v 3 


4 


5 


9. Metho'ds of monitoring child's progress 


1 ? 


3 


4 


\ 5 


10. Accomplishments of program < - 


1 2 


3 


4 


5 



C. Perceived Change In Child 



VaazcUoka: Thz Ptiztchool Vttoqfmm Is dtAijgnoil tn Houofnp ,jn^ rhtM't dkittt in thz 
thz Itvzt oh pnoQtuu>6 youA child ha* made by cJjidUng onz riumbeA htwm 1 


oJiQjxs ti&tzd bttow. 
Urn) to $ [high). 


Iron, zach a/tea, 


plejtxAz ehzck 


* 

Low 


LEVEL OF PROGRESS 




. Hiqh 


1. Speech and language skills X 1 


2 


3 


4 


5 


2. Physical and motor skills 1 


2 


3 


4 


5 


3. Social skills * 1 


2 


3 


4 


5 


4. Self-help skills. 1 

• ■ . 1 


2 


3 


4 


5 
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SECTION III: Open-Ended 'Questions 
' 1. What do you see as the major -strengths of the Preschool Handicapped Brogram? 



„ 2. What do you see as the major weaknesses of the Preschool Handicapped Program? 



3. Would you recommend any changes in the Program? If so, what changes and why? 



4. Would you recommend this program to another parent of a preschool handicapped child? 



TIT 



THANK YOU FOR YOUR COOPERATION! 



(ERIC 14 



/. 



No 



" Parent Signature 

15 
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I. FOCUS 

Certain decisions, if made before the questionnaire is developed, will 

make the construction of the instrument easier, improve the responses that are 

* 

obtained, and enhance the results that ultimately are available. 

A. The specific purpose(s) of the questionnaire should «be determined* Ques- 
tionnaires are used to gather different kinds of information. Ttia^eforei 
before questions can be developed, the designer should determine. whether 
to collect facts; test the knowledge of the respondent; discover 
respondents 1 opinions, beliefs, or attitudes; or any one oV these in 
combination. The questionnaire used as an illustration in Figure 1-was 
designed to gather information from parents for the project's evaluation. 

" This definition of purpose, however, does not provide sufficient detail to 
help understand or construct the instrument* A more specific purpose of 
the questionnaire might be "to gather information from parents regarding 
their involvement in the project and their reactions to the project's ser- 
vices for themselves and their children." * 

B. Identify the persons who will tfse the results and identify their reasons 

for wanting them. Identifying tffe individuals or groups who will use the 

1 4 * 

results of your questionnaire may fo^p determine the purpbse of the 

questionnaire and guide your choice of items. If, for example, the 

project s£aff will be using the results to improve their efforts, you 
r 

could ask questions about specific components of the project. On the 
other hand, if a^governing or funding agency will be using the results, 
questions regarding the overall impact of the project might be asked. 



V 



Identify the major categories of information you seek . Identifying cate- 
gories of information needed means clarifying the content of the questions 
that will be developed. A content outline will help you define the' pur- 
poses and scope of your instrumete. For our sample instrument, the con- 
tent outline might be: 



)ntent|i Outline 
Parent Questionnaire 

A. Demogrdphicf Information 

1. Length of time chtld has been in the program ' 
. 2. Type of services 1n whl^ch the chi Icfwas. enrol led / 
3 4 Parent volunteer activities 

^ B* Parent Involvement 

1. Source* of information regarding the program 

2. Type of involvement in the program 

3. Reactions to parent activities ■ ' 
• ■* \ 

C. Services for Children 

1. Knowledge of program ^for children 

2. Satisfaction with program for children 

3. Perceived change 1n child t 

D. General Opinions " i 
1. Strengths 

2* Weaknesses 0 m ' 

3. Recommendations 

After the instrument has been developed, the" Items -should be compared 
with the content outline to see if allareas have been included. 

Identify the specific group(s) that will answer tjie questionnaire . Be 
specific so that: (a) the targeted groups will receive the questionnaire; 
(tjj the^results can be described as representing a particular group; and 
(c) the characteristics of the group can be taken into accounfrsghen you^ 
construct the instrument. The questionnaire in Figure I obviously will be 
sent to parents. Further specifics might a»lso bjp helpful — e.g., only 



to mothers, only to th£ parent who cares for the child mo$t of the. time, 



. etc. 



E. Identify the type of information ^you see,k . Preliminary decisions regard- 
irig the most usefjrPtype of information will help you later to select ap- 
propriatg items. Some questions (such a^s those regarding length of time 
the child has been in tfre program) might best be answered with actual num- 
bers. If you wnat to obtain opinions that can be compared, use a standard 

item with^pecif "fecf responses. ^Finally, if you desire individual opin- 

<* ■ 

tons, use 'questions which permit a variety of responses. 

• ■ c 

When these preconstruction detisions have been made, a great deal of in- 

° * (» 

formation will be availably for actual construction of the instrument/ ^ 

• / 



II. TITLE 



> , .9 * 

Ev^Py instrument requires a title to summarize briefly the content of the^ 

s 

instrument. Three criteria shbuld be considered: 

-at 

1 • 

A". The title should reflect the content of the instrument . This can be done 

in at feast two ways. Fjjjrst, the title simply can reflect the content of 

the items. p For example, if a person has constructed a .questionnaire on 

attitudes toward the handicapped, an appropriate title might be: "Survey 

of Attitudes,* Regarding the Handicapped." A title! also might reference the 

■ 

""content of the items and the major type of item in the instrument. For 
„ example, if you develop an *i nstrument to assess a teacher's ability to 
teach, and all or most of the items are 5-point rating scale items, you 
, might appropriately ^itle your instrument a^Teacher^ Rating Scale." It is 
important that the title does not mislead the respondent. 

A^title also may summarize the purpose of the instrument: "Pacgjgt 
Evaluation of Child Services." 

3. The title should be conciW and to the point. Long titles tend to dis- 
tract and divert the respondent from the task of actually filling out the 
questionnaire. The title should allow the respondent to grasp the scope 
of the instrument quickly without becoming, bogged down with specifics. 
The titles above are good-examples. A wordy and, consequently, bad title 
for a political attitude survey would be: "A Survey of Attitudes, Reac- 
tions, and^jQJxservations of Legisl^ofs Regarding the Handicapped in the 
U.S. Dur.ing the Past Ten Years." g 

; ' . ' ;/ ' if 

C. The title should be written in, plain English . Complex or unrecognizable 
words or phrases in the title of the/instrument might discourage potential" 

119 



S" 1 1 18 

% 

respondents, so eliminate jargon and acronyms. In general the title 
should be easy to understand a/id informative. Your well-designed instru- 
ment may remain unanswered i f pebple are alienated or confused by the lan- 
guage in the title. ^Writing the title is usually best done after con- 
struction of your instrument, so that it more accurately^ref lects both 
content a/id/or method of eliciting responses. 



\ 



/ 



i 
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III.' INTRODUCTORY STATEMENT * ' " . 

t 

An introductory statement should contain information about the 
instrument's purpose and how the data will be used, anonymity, and motivators 
for respondents to complete* the questionnaire*. This introductory statement 
usually is found in one or two* brief paragraphs following the title of the 
instrument, or in an accompanying cover letter. * 

Consider the following: ' 

~ ... * 

A. The introductory statement should include a brief summary of the instru- 

' : * * 

ment's purpose . This statement indicates now an(£ by whom the information 

will be used and tells respondents the kind of project with which they are 

involving themselves. The statement should be brief and straightforward, 

with no intentional or unintentional bias. 

Looking back to Figure 1, the introductory paragraph states that the 

purpose of the instrument is to contribute to the evaluation of a program 

for preschool, handicapped children. More specifically, the information 

gathered* wi I I be used to describe how the program is functioning its 
• » 

accomplishments, constraints, arid concerns. 

As mentioned previously, the purpose and use of the questionnaire 

♦ * > 

should be written before actually constructing the instrument. The pur- 
poses,* if more than one exist, should be^ consistent. Inconsistency can 
cause respondents to lean toward noncommital responses, which result in 
data that contain little real information. The statement of" purpose also 
Treminds the instrument constructor to develop items within a certain scope 
--i.e., information pertinent to the use of the instrument. A side 
benefit of providing the respondent with a description of how the 



information will be used is that it provides each respondent an 

1 ' % 

opportunity to, gauge the face validity 1 of the instrument. 

r ; * 

B. The introductory statement should include an appropriate statement con- 
cerning the respondents' anonymity . Tim statement should jilake explicit 
whether or not an individual's cimplfcted instripefvt*^wTl II bemused and be 
made acjessvble to others or only as part of an'aggregation.S Obvious lys 
"digging respondents negative comments -or suggestions migh^jiave serious 
repercussions for them and thereby temper their responses. Furthermore, 
it is illegal to elicit information under pretext and use the information 

/ for another purpose. In Figure 1, the introduction states, "Al fc^jesponses 
will be held in strict confidence, and only summarized data will ^.'pre- 
sented in subsequent reports." Exclusion of this statement might c a.u^e 
respondents to think that their comments, individual ly identified, jm^^af- 
fect services to thm or their child. / \\ 




C. The introductory statement ,shou Id use a "motivator" to encourage the V 

>j ■ + 

respondent . Examples of frequently used motivators include promising re-U 

' " ^ ' \ 

spondents *a copy of the results of the purvey and making respondents feel 

as though they will have an impact on an organization. More expensive 

motivators include free tickets, money, and access to generally inacces- 

sible places (i.e., reserved parking areas or .di scottots o 1 n purchases). A 

well-known gardening magazine recently promised respondents a free plant 

t "J 
• if a EOO-item questionnaire was completed. Often, outside motivators » 

t 

exist which ensure replies —e.g., withholding a person's paycheck or 
making the response a" general expectation. 



iFace validity is the extent to which a respondent judges the instrument-to 
measure what it purports to measures 
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Motivators take a variety of forms.* The important considerations, are:/ 
first, can you afford the motivators, .and second, does the pressure o'f the 
motivator potentialJy bip the responses? If a motivator works, ^nore re- 
spondents repty, and the tfata gathered usually will better represent the 

" * ^ 

/ sample or population surveyed. 2 

D % The introductory statement should "be written at the language level of the 
, respondents . Here, as in the tiyie, and in all other sections of an in- 
strument,' the language used shouljj^be understood easily by the respondents. 
This Seems obvious, but remember, if you do not know the reading level of 
the respondents, ybu may unwittingly write items at your own . level ... Jhe 
problem can easi ly be remedied by administering the questionnaire to a 
subsample of respondents and questioning them verbally tb check their 
understanding of the terms and expressions. The instrument designer and 
the respondent must share these mearfings or the instrument will not mea- 
sure what it purports to measure. 

.. One last word concerning whether introductory statements are required in ( 
all instruments.. In many situations the factors listed above are implied. 
If any of these areas are not clearly implied or stated elsewhere, a statement 
pertaining to each should be included. 



2This is not true if the people responding consistentfy represent one portion 
of the population rather than another. NFor example, if your population po- 
tentially had an equal number of Republicans and Democrats, and Democrats 
cOmri-sfently filled out the* questionnaire mope tftan Republicans, your overall 
results would contain that bias. Inc*4asing the sample responding would not 
necessarily eliminate the bias. 
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IJ. DIRECTIONS . \ 

v Basically, directions come uhder two headings: general directions /relate^ 
1 to the total questionnaire; specific directions deal with individual sectiorvs 
of the instrument. At this point, only general directions wil 1 be considered 
(specific directions will be explained in other sections of this book). 

Consider these important factors when you ; write general directions for 

your questionnaire: 

.J ' \ . ; 

s \ ■ ' 

A. Directions slrould be complete, unambiguous, and brief . 

B. Directions should be written at the language level of the respondents . 

C. Directions should tell respondents to whom they should give the instrument 
^ once they have completed it . In some situations persons who adminis- 
ter the instrument may hand out the questionnaire and tell the respondents 
to return it when it has been completed, this case' written directions 

♦are not required. If, on the other hand, the respondent receives the 
questionnaire in the mail, clear instructions are- necessary. A self- 
addressed, stamped -envelope facilitates easy return. This procedure adds 
to the cost of the survey, but it generally increases the response rate. 
Further, it is a good idea to give the respondent a deadline to complete 
the instrument e.g.^ "Please return the completed questionnaire within 

the next week." This time specification can make the, respondent aware of 

„ • V 

the importance of returning the questionnaire immediately. Another exam- 
ple of a time specification might be: "Only those responses received by 
October 1 will be used in the data summaries." A clear time deadline par- 
ticularly is effective when respondents are motivated to 
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V " :, ' ' ' ' 

| answer a questionnaire because the results of the instrument directly af- 
feet them. ^ a * * « * . 

WJaen^the questionnai re i s 'accompanied by a separate answer sheet, the gen - 

P • . . . » 4**- 

; \eral directions^hou Id specify* how the respondent should fill out the •' 
Answer sheet. , A 



The general -jltrections ^should explain* to the respondent how to deal with 
items whicfr are not applicable . Does the respondent leave those answers* 
blank? Is an appropriate spjee provided? Is the respondent expected to 
respond to all items? Ensure that all respondents respond in a similar 
way to the^ori^plicable items, and the results will be easy to interpret 

and analyze. 4 ■ . - 

+ ■:. / 



F. Tire general directions should specify the approx imate ^arrfount of time re- 

~ : : : f ~~ — 

quired to complete the instrument . Jhis criterion is particularly appl,^ 
cable for a long, questionnaire. Time approximations can help alleviate a 
respondent's concern that the questionnai re ,wi 1 1 .take too long to complete. 

/^s is the case with the introduction, place the general directions accord- - 
ing to the structure of the questionnaire. The beginning of the instrument or 
in the cover letter are two^a^propriate places. 

A1 1 the 'criteria stated; above are not required for every instrument, but 
they all should be considered to determine which are appropriate for a given 
questionnaire. 
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V. DEMOGRAPHIC SECTION 

... * 

In almost every situation, whether. a person is developing a questionnaire 
for an evaluation study or to gather: input, it is useful to obtain certain 
demographic information. For example, in Figure 1, the demographic variables 
are found on the first and second pages. They ask for information regarding 
the services provided to children by the program and the parent services in 
which the respondent participated. 

Though demographic variables differ according to the type and purpose 
of the questionnaire, some general guidelines can be fo Mowed. 

A. The demographic information shou Id be limited to only those variables that 
will be used to 'answer specific questions/ All of us are familiar with 
questionnaires that begin with lengthy demographic sections which e 1 i c i v t 
information that seems unrelated to the overall purpose of the question- 
naire. To avoid this problem, the person constructing the instrument 
should consider how the information gained from any given demographic item 
will be used in data analysisf. - # If uses. for an item in particular or in 
comparison with other items -- cannot be identified, then it should not be 
inc I uded. 

Demographic information which will help you understand the data is . 
important. v To select appropriate demographic* information, review a list 
of standard characteristics for the insights they may give to the data. 
The list might include^:' age, race, sex, and type of handicapping con- 
dition of th§, child — just to mention a few. Identify the critical 
(information y6u need and make sure you get data on only those variables. 



The demographic section should reveal the relationship of the respondent 
to the object of measurement . One important category of demographic data 
sometimes is overlooked and relates to the respondent's judgment^ about 
another person's performanpe. For example, parents are often aste^d ques- 
tions about the project staff, or project^staf f are asked questions about 
other service providers or agencies. In these situations, the respondent 

0 

must divulge the amount of contact with the person or thing being rated. 
If a parent is rating a center-based teacher, for example, determine the 
extent to which the parent has observed ttfe teacher in the center. Cer- 
tainly, data from a parent who has observed a teacher should be viewed 
differently from data given by. a parent who has not observed a teacher. 

Certain items in the demographic sections should be optional . Though this 
criterion rarely is important, it has specific implications for situations 
where decisions about individual respondents wi 1 1 be made. For example, 
if you are polling potential applicants for an advertised position, fed- 
eral law prohibits considerations based on sex, race, or religious prefer- 
ence. This problem can be avoided by briefly explaining why. the info.rma- 
ti orris needed and requesting that the respondent provide answers t^o these 
questions. 

Language in the demographic section should be appropriate to the language 
level of the respondents. 

Demographic items shoulcl be grouped^bogether . the positioning Of demogra- 
phic items 1s not Important, bui it is usuaNy a good idea to group them 
together. In Figure 1, the. demographic items come at the beginning of the 
instrument. In fnost cases, the demographic section follows the introduc- 
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tory statement and the general directions, though sometimes it is located 
at the end of the questionnaire. 

Sometimes tfemog^raphic information may be left out of the questionnaire 
because that ipformation can be obtained elsewhere. For example, if you were 
conducting a survey of the incidence of handicapped young children in the com- 
munity, you probably would not have to ask each respondent the incidence of 
low-birth-weight babies they encounter; this information could be obtained 
easily from some other source such as hospital records or birth certificates. 
Obtaining appropriate demographic information from other sources accomplishes 
at least two purposes: first, you relieve the respondents of a somewhat tedi- 
ous task; second, you ensure the compatabi lity of the demographic data across 
your respondents. Another illustration of this procedure might involve a" sur- 
vey of children who have left a program. If the program has kept an up- 
to-date fyle on the children, demographic information could be found in the 
file rather than from the respondents. 
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VI^RITING ITEMS 

Types of questionnaire items include: rating scales, checklists, short 
answers, yes-no questions, open-ended questions^ ranking questions, and sever- 
al others. Each method presupposes certain assumptions, uses, advantages, di- 

>> 

sadvantages, construction, directions, and data analysis techniques. This se- 
ction will discuss all such points of interest. If you have not selected your 
item type or wish to confirm your selection, read the entire section for help. 
If you have chosen your item type already, read the discussion which applies 
to that type of item. * 

Before describing specific types of items, several comments are in order* 
Two parts are inherent in any item type: the stem elicits the information; 
the response is what the respondent does to complete an item. Generally, the 
responses are supplied freely by the respondent or are selected by the respon- 
' dent from a list provided by the instrument designer." 1 In the construction of 
any instrument, various types of items can and should be used, depending on 
the information to be gathered and the eventual use of data. Jhe developer 
should, therefore, consider the best way to obtain the required information 
and use as many of the various item types as are appropriate. ^> 

A. Rating scales . A rating scale is a device to indicate ttie degree to which 
a certain. trait or attitude is present in a person, place, or thing. A 
rating scale is made up of two parts: a stem and a response set. The 
stem is the stimulus which elicits the response. The respondent selects 
an answer from the response set which contains categories which may or may 
not be numerical. 

The following assumptions are made when one uses a rating scale: 



** The 'attribute can be sealed a \pr\g some continuum. For example, aa 
item rating the effectiveness of a training workshop assumes that 
the attribute "effectiveness" can be scaled from ineffective to 
highly effective, 

** The respondent is presented with a continuum that is isomorphic 
to his/her internal continuum. In other words, the continuum ap- 
plicable to the actual attribute is consistent with that within 
his/her self. Perhap^the best example of this would be one in 
which the trait exists but there is no recognition of it. If the 
particular trait to be rated were greenness and the ratings were 
to be made on a scale between blue and yel low, then a colorblind 
person would not Yiave an 'isomorphic continuum on which to rate the 
particular attribute. # 

** Th6 respondent can consistently discriminate between different 
s levels of the continuum. 

Empirical testing of these assumptions would be taxing to say the 
least. .Importantly, however, each potential constructor should consider 
whether or' not these assumptions are pertinent. If you cannot make the 
assumptions\listed above, employ alternative item types. 

This paper will discuss three of several formats for writingJ!,ratinqs 
scales: numerical (currently the most popular), graphic , and Likert . 

A numerica I sea le requires the user 'to indicate a particular number in 
a stem which may or may not be associated with verbal description. Figure 1 
2 shows several different formats for numerical scales. 

These examples represent only a small variety Qf numerical rating sca- 
les. The items in Figure 1 are additiona I* i I lustrations. Though numerals 
do not appear in most cases, numerical scales will produce ordinal or nom- 
inal data^. The choice - of one of these types 'depends largely on the pref- 
erences of the instrument designer --who has knowledge of the respondents 
and the %Qta I structure of the instrument. 

Graphic scales present a stem along with -a response set containing a 
line on which the user designates a response. Examples of the graphic 
type of scale might look like this: , . . 



Figure 2 



Examples of Numerical Rating Scales 



Example: Assume that you have been Instructed to develop an instrument to 
determine parents 1 perceptions of a parent/child activity guide you have developed. 
One of the variables of Interest may be the parents 1 perceptions of the readability 
of the guide. Y * 



1. Rajte the readability of the activity guide compared with other guides for 
parent/child activities you have read. 

( ) ( ) ^Jf*) ( ) ( ) 

very difficult difficult about average^ easy to very easy 
to read to read to read - J* read to read 



2. Rate thfe readability of the activity guide compared with other guides for 
parent/child activities you have read. 

( ) ( ) ( )' ( I ( ) 

1 2 3 4 5 



5. Rate the readability af the activity guide compared with other guides for * 
parent/chil.d 'activities you have read. 

" ( > ( > ( ) f ) ( ) , 

very difficult about average very easy 

to read to read to read 



4. Rate the Readability of the activity guide compared with other guides for 
parent/child activities you have read. 

a . very difficult to read 
. b. difficult to read 



_c. about average to read 
_d5 easy to read 



_e. very easy to read 



5. Rate the readability of the activity guide compared with other guides for 
parent/child activities you have read. 

j— ~i r~— 1 i 1 \ 1 i ~ i 

very difficult very easy 

, to read 1 ; to read 



6. Rate the readability of the activity guide compared with other guides for 
parent/ child activities you have read. 



very difficult very easy 

to read 'to read 
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Rate the readability of the activity guide compared with other 
guides for parent/child activities you have read. 



very difficult 
to read 



) 



difficult to 
read 



about 
average 



easy to 
read 



very easy 
to read 



Rate the readability of the activity guide compared with other 
guides for parent/child. activities you have read. 



r 



very difficult 



about 
average 



very easy 



Likert-type scales are somewhat different from numerical or graphic 
scales ~ each set of responses, ranging from "strongly disagree" to 
"strongly agree," is invariant and not specific to the stem. As an exam- 
ple, the Likert-type item would require a modification of the stem in 
order to be comparable to the previous example: p 

The activity guide is very easy to read compared with other guides 
for parent/child activities I have read. 



( )■ 
strongly 

disagree 



( ) 
disagree 



( ) 
undecided 



( ) 
agree 



( ) 
strongly 
agree 



The activity guide-is very easy to read compared with other guifles 
flor parent/child activities I have read. 



^ ( SD } 



( ) 
D 



( ) 
U 



( ) 
A 



( ) 
SA 



Obviously, these examples /how a variety of rating scales which elicit 
the same information. When you have made the necessary assumptions and 
have chosen rating scales, consider these criteria: 



9 

ERIC 



32 



33 



The stems of rating scale items should be written unidimensional 1y. 
I This means that only one attribute or trait shoi^d be described in the 
stem. In the first two sets of examples, the attribute to be rated 
^g|feadability. Furthermore, readability was limited to comparison 
o1| this guide wit^ others. Thi$ eliminates the possibility that a 
respondent will compare the readability of the activity guide to other 
types of reading matter. It also allows the person reviewing th^data 
-to make a straightforward interpretation. Here is an example of an 
item which is not unidimensional : 

Rate the effectiveness of the activity guide and the assi starfee q, 
provided by the home teacher in enhancing your child's develop- 
ment. 

( V ( ) ( ) , ( ) ( u 

extremely somewhat neither somewhat extremely 

ineffective ineffective effective .effective effective 

nor 

ineffective 

Interpretation of data from this question would be difficult, 
since an answer of "neither effective nor ineffective 11 could have 
several interpretations: . 

a. The activity guide was effective while the assistance from the 
home teacher was ineffective. 

4 

b. The assistance from the home teacher was effective while the 
activity guide was ineffective. 

c. Both were neither effective nor ineffective. 

Though this and many of the following criteria may seem fairly |^ 
obvious, the good instrument designer will apply each criterion to 
every item to ensure thatfal l v items are unidimensional. The problem 
presented in the example t above could easily be resolved by splitting 



the item one question concerning the effectiveness of the activity 
guide and th§ other concerning the effectiveness of the assistance 

j ' ■ . • ^ . 

provided by the home teacher. 

* 

3 ■ . 

The response sets td rating scales should be written unidimensional 1y . 
Thi§ criterion is related closely to the previous one and 1 - relates to 
the the response set of any item. To ensure unidimensional meaning in 
the response set, refer only to the attribute described in the stem. 
In our previotis example, we see that though the stem had two things to 
be rated, the response set was unidimensional (i.e., effectiveness). 
Critferia 1 snd 2 should be considered jointly for each rating scale 
item employed. 

• i * ' * 

The response set of a rating 'scale should be tied logically to the 
stem. For example, if you ask the respondent to rate the readability 
of a guide, the following response set would be inconsistent with the 
stem. 

( ) ( ) ( ) * ' j> ) ( ) 

definitely probably don't know probably definitely 

riot not 

The level of specificity of the stem item should answer the user's 

\* / 

needs. For example, if the purpose of, an item is to determine gener-* 
ally whether one teacher is m^re effective than another, the question 
could ^be phrased, "Rate the overall effectiveness of the instructor. M 
If the purpose is to provide feedback to individual teachers concern- 
ing their various teaching abilities, a general item would not be par- 
ticularly useful. A series of rating scales assessing specific 
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• * 

teaching acts -- abi lity to plan and work individually with children 

* * - * - 

or lead play -- might prove to be mojfe informative^ 

5. Use the rating sea le iten/only^when a more direct method is not avail - 
able . Rating scales /dfenera I ly, are overused, and the extent to which 
their assumptions can be met is difficult to document. Therefore, 
rating scales should be used only as a last resort or with more relia- 
ble and valid item types. Section 3 (previous page), Readability of 
an Activity Guide, represents a good example of the misuse of rating 
scales. In our examples of items rating readability, analyzing the 

'words used in the activity \juide by grade level and comparing that 
with the actual reading level of the participants is a more direct way 
m of getting the same information. 

<v ' . • 

6. Directions for using the rating sc(ale should be included, along with 
appropriate exampFqs . Unfortunately, many questionnaires ask for rat- 
ings atong a certain continuum, but do not specify how those ratings 
should be made. For example: 

/_ / V J I / /_ / T / 

I hate it . I love it 

Here, the actual rating task is unclear. Should you place an X 
between the lines? If so, what does an X in the middle space mean? t * 
Should'you circle one of the oblique lines? Obviously, the designer 
of the instrument had a clear idea of the appropriate response but did 
not make that clear to the respondent. To ensure that all respondents 
know how to respond appropriately, the designer should (1) provide ex- 
plicit directions of rating procedures and (2) prov^e an exfltople. 
Let us look again at the examples in the introduction to tlws section. 
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Obviously, slightly different directions are in order for each 
individual case. ' 



a. Directions for a Numerical Racing Scale 

You are to rate the following items by placing a checkmark .'in the 
appropriate space under each item. For example, if you feel that 
thet guide i$ as readable a£ activity -guides for parents and 
♦ children, you would place a check in the space labeled ''about 
average to read," as shown below. 

Rate the readability of the activity guide compared with other 
guides for parent/child activities you have read. 

( ) ' ( ) \ (Vl ' ( ) \ ) 

very difficult difficult about average easy to * very easy 
to read to read to re*d read to read 



b. Directions for a Graphic Rating Scale ,\ 

1. In the following set of items, you are to rate each item by 
placing an "x" over the set of words which best describes your 
feelings. For example, if you feel the activity guide is more 
readable than guides for parent/cty'ld activities, your * 
response- would look like this:?* ^ K £,\ 

Rate "the readability of the activity guides compared with other 
guides for parent/child activities you have read. 



very difficult difficult about average easy to very easy 
to r£ad * tti read & to read read to read 

2. In the following set of items, you are to rate each item by 
placing a check in the approprwte space. , As you wil l noteN, 
only the erfds of the scale are marked. Your task, therefore^ 
is to consider each item on a five-point continuum^ decide \ 
which line on the continuum corresponds with your feelings, 1 
and place an "X" on that line. For example, if you feel that 
the activity guide is a little more' difficult than av&rage * 
your response would look like tyns: ^ \ 

Rate the readability o/ the activity guide compared with other 
guides for parent/child activities you have read. 



t 



very difficult , "very easy 

to read ■ \_^J> Q read 

* .... 
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c. Directions for a Likert Scale. 

• In*the following set of i£ems, you are to decide whether or not 
you strojigly agree (SA), agree (tf), are undecided' (U) , disagree 
* >(D), 4 or strongly disagree (SD) with each of the statements listed, 
and indicate your preference by circling the appropriate letters. 
For example, if you agree with the statement that thfe activity 

guide is very easy to read, you wou Id circle "A," 

" «• * 

The activity guide is very easy to read compared with other guides 
for parent/child activities I have read. 

SD D U A ^ SA 

The 'directions above, though not comprehensive, suggest the types 
of statements that ensure understanding by the respcmdent. A The impor- 
tance pf these directions should not be underempjiasized, sinee the 
* validity of the data depends on the extent to which the subject under- 
stands How to respond. Directions, it^general, should be as brief as 
possible. If you are sure that your respondents are £ami liar with 
your rating scale, no examples are necessary, however specific dirge- 
tions for any set of simi lar items are advisable. 

7. The language used in A rating scale stems ancj responses should be writ- 
ten at the respondents' level of understanding . 

8. " The 'type of rating scale format selected should be easily understood 
by a 1 1 resppndents . If a question arises as to whether or not a par- 
ticular scale is appropriate to a given group, the best procedure is 
to administer several types of scales to a sample target group. You 
should question them as to the procedure with which they feel most 
comfortable and which best allows them to express their opinions. 
This procedure, though time-consuming, will help/ensure valid results. 
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Rating scale items Should' not elicit biased responses. Item stems and 
responses should be written in such a way that the user*will not think 
that certain responses are preferred. Example: 

• * 

Rate the extent to which you enjoyed the immoral movie "Peyton 
•Place," * 

< J.. <• ) ( * ( ) '•'<■> 

one of the - ^ one of the 

worst I ^ best I 

have seen ** have seen 

Clearly, the bias in. the stem suggests that a person should rate* 
this item low rather than high. A quick perusal of item stems by a 
colleague or other qualified person should help eliminate this problem. 

The response sets for a rating scale should present choices along sim- v 
ilar degree continuums, this 'criterion probably is the most contro- 
. versial. Manynjuestionnaire writers argue that all positive or nega- 
tive poles similarly situated on a page encourages "patterned" re- 

• " ' i 

sponses. That is, the respondent simply may mark all. items in the 
same way without thinking about them individually. Our experience in- 
dicates,^^ the practice of reversing response poles is very confus- 
ing to users and therefore tends to invalidate their responses. Po- 
tential bias may*be counteracted- by structuring the rating scale items 
in, such a way that response sets are specific to only. one question 
and, together with the stem, comprise a single entity (see Figure 3). 
Each stem and its response set graphically displayed as an' entity also 
is useful when "response sets are the same across items. If you can * 
establish that users do not respond similarly to items (by noting pre- 
vious questionnaires or answers to a short pretest), then choose the 



Figure 3. Example of Individual' Layoutfor Rating Scales 



Interest in children 

*( I ( ) 

always 

enthusiastic 



( ) 
sometimes 
enthusiastic 



( ) v: ( ) 

never 

enthusiastic 



V 



Availability to parents 



( J 
always 
available 



t ) 



Personal appearance 

( ) ( ) 

always well 
groomed, clothes 
neat and clean 



( ) 
sometimes 
available 



( ) 
somewhat 
untidy at 
times 



t ) ' (< ) 

never 
available 



( ) ( ) 

slovenly, 
clothes untidy 
and urjkempt 



Stimulates intellectual curiosity 



( ) 
inspires 
children to 
i ndependent 
effort 



( ) 



( 

occasionally 
inspiring 



( ) ( ) 

destroys interest 
in activity, makes,, 
participation 
repulsive 



matrix format (see Figures 4 and 5). The primary advantage of the 
matrix format is its use of space; maximum information can be obtained 
in minimal ; spiace. 



• * * 

Figure 4. 


Matrix Layout for Rating Scales 


\ ■ 


Rate your child's teacher 
average to above average, 


on the following attributes from below 

by cirtling the appropriate number. 

■ 






way below 
average 




way above 
average 


Interest in children 




1 2 


3 


4- 5 • i 


Availability to parents 




1 2 


3 


4 5 


Personal appearance 




1 2 


3 


4 5 


Stimulating intellectual 


curiosity 1 2 


3 


4 5 



(1 



Figure B, A. Variety of the Matrix Layout 

Rate your child's teacher on each of the attributes below by placing 
the appropriate number on the space provided. 

1 ■ way below average ^ 

2 = below average 

3 ■ average 

y 4 1 ■ above average 
5 = way above average 

l.v Interest in children 



*2. Availability to parents 
_3. Personal appearance 
4* Stimulating intellectual curiosity 



V 
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Another way to avoichTesponse bias is to intersperse negative and 
positive stems, while keeping the response sets the same. The is 
appropriate particularly with the Likert-type scales. For example: 



This parent/child activity guide is one of the most readable I 
have encountered, ; 



( ) . ( ) [ ) ( ) ( ) 

strongly disagree , no opinion agree strongly 
disagree , \ * agree ^' 



This parent/child activity guide is one of the least readable I 
have encountered, * 



( ) ( ) ( ) , • ( ) ( ) 

strongly ' disagree no opinion agree strongly 
disagree agree , 

11, Avoid using global terms in response sets. Examples/of global terms - 

include: best to worst;" all of the time to none of the.-time; and high 

* 

score. to low score. The major problem with global terms is that their 
interpretation lies with each respondent. For example, rf instrument 
^ designers were interested in how often the respondents went to the 
movies, they could ask the question ih the following ways: 

Rate the extent to which. you attend movies. 

( ) ( ) ( ) ( ) ( ) 

never , infrequently sometimes often ' regularly 
attend ' \ attend 

A better way to elicit information on this same variable would be: 

( ) ( ) ( ) ( ) . ( ) 

'never once or twice once a once a week more than 
v ' • * a year : month once- a 

week 
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The latter construction helps ensure that every person is using 
the same continuum of response, and- providing' information, which is 
easily interpreted. 

12. Rating scales should include from three to seven categories . To 

> - 
decide on the number of response categories to use for a particular 

Item, consider the trait being rated and the extjSfct to which individu- 

al categories can be identified. Ask yourself these questions: Is It 

possible for the respondent to mafce the discrimination demanded of in- 

dividual categories? Should an even or odd number of categories be 

provided? The advantage of an even number of categories is that th^ 

respondent must be either positive or negative about the attribute 

(not noncommittal). The advantage of an odd number of categories is 

that the respondent can be noncommittal . 

13. Choose the format for your rating stales that corresponds to the so- 
phistication of respondents. Generally, two basic formats exist. A 
matrix format is shown in Fig^es 4 and 5, and an individual format is 
showrt in Figure 3. All examples are based on the same item stems. 

The matrix arrangement al lows the instrument designer to cover 
more variables in less space. However, it necessitates the use of 
general terms in the response set, since it fhust be appropriate for 
many different stems. Further, it may allow the subject to respond to 
V all items in the same fashion rather* -than responding to each item in- 
dividually. For example, if parents liked a teacher, they might rate 
the teacher high on all items without reading each itfcm specifically., 
Generally, the matrix display structure should be used only with 
respondents who are sophisticated in the use of rating scales. 
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The individual format is effective with a leas sophisticated 
group, since it forces respondents to read each item individually and 
offer's response se^s specific to individual items. 

14. Allow for a "not applicable" response when appropriate . Respondents 
may not have had the opportunity to form perceptions about specific 
traits the questionnaire assesses. So, a "not applicable" response 
should be available. However, this option can cause problems in later 
aggregation of data. Use the "not .appl icable category" only when 
appropriate: example — If the, presenter in workshop A and B used 
audiovisual equipment, but none' was used in workshop C, then attendees 
of workshop C should not have to rate'the effectiveness of audiovisual 
materials. ; 

15. Choose an analysis strategy appropriate to the data produced and the 
uses for which the questionnaire was designed. Once you have complet- 
ed the construction of your questionnaire and administered it tjl a set 
of respondents, you are ready to analyze and report the results. It 
i<s always a good idea to summarize ar tally the responses before deci- 
ding on specific analyses. A careful review of the actual results of- 
ten will indicate the are^s for aftd types of analyses that would be 
most useful. Then the data can be analyzed in a variety of ways de- 
pending on the assumptions made about the scales themselves. If the 
designer assumes the scales are of an interval nature; all types of 
parametric statistics are appropriate (when their assumptions are 
met). If these assumptions cannot be justified, then nonpar^net^tc 
statistics should be used. Further discussion of these^topics can be 
found in texts on basic statistics. . . 
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B. Qualitative selection items. In the preceding section, Rating Scales; we 
.discussed selection of responses based' on some continuum. relating to the 
stem. In this section, vie will discuss responses that are not on a con- 
tinuum. For example, if. you are Interested in the sex of your respondent, 
you have the stem "sex*" and the responses "1, female" and "2, male." An- 
other example is the following: 



What is your position in the program? 



J Administrator 

"| Teacher 

_ V 

J Occupational Therapist 

J Social Worker 



I | Other 'Vl 

-* » . 

Tiie purpose of this type of selection item is to elicit qualitative •* 
information from the respondents. For example, job classification, sex, 
or race are more suited to this type of item than number of children, 
hours of participation in the project, etc. Responses to the former will 
be qualitatively different, whi le responses to the latter wi 1 1- be quanti- 
tatively different. Qualitative response categories also are used to 
elicit demographic information; all respondents can select from the same 
set of response categories. This type of response also is appropriate 
when all or most of the response categories are known.' Finally, this type 
of, item requires no judgment in the, coding of responses, since all inter- 
pretative judgments are made when the instrument is constructed. 

At least six rules should be applied when constructing items of this 
type. . - r , 

1. State clearly and unambiguously the stem and responses. 
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Use language appropriate to the respondents * 
State the stem untdimensiqnally. 

Write an exhaustive response set. When you write responses to an 
item, include all possible choices. For example, if you are surveying 
the positions of the respondents, you may include the following selec- 
tion items: 

k 

Identify your position. 

y ' • 

r 'J 

| | Admi ni sthator 

| " 1 Teacher 

1 " 1 Occupational Therapist 

\ You have not been exhaustive, since there is no place for the 

sbclval worker or teacher to check. This problem can andlshould be 
j * 

avoided by including an "other" or "miscellaneous" category, as demon- 
st rated below. 

Identify your position. 

I I Administrator 

1 ' I Teacher 

| | Occupational Therapist / 

| " | Other (please specify) 

The "other" category makes the item applicable to all respondents 
and finds \out who or what the other categories include. This latter 
benefit is of particular significance both in qualifying your results 
or in future construction of 'similar items. 
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5 » The respon se categories should be mutually exclusive. Mutual exclus- 
iveness means that the .categories do not overlap. Numerical 
categories most frequently overlap. An example: 

How much time do you spend working with your child on project 
activities during the average week? (check one) 



"| 0 - 1 hours 

I 1-3 hours y 



|_ | 3 or? more hours ••' • ♦ - 

<# 

The parent who estimates either 1 hour or 3 hours must choose a 
•category arbitrarily. Several alternatives are available to ensure 
mutual exclusiveness in responses for numerical data. First, don't 
use select ion- type item| when numerical categories are the required 
responses. You may use instead a*n -open-ended type of item |£o be dis- 
cussed more fully in a later section). Second, if numerical categor- 
ies are to be used, ensure mutual exclusiveness by naming limits that 
do not overlap. The above item would then read: 

How much time do you spend working with your child on project 
activities during the week? (check one) 



Less than one hour 
Between l and 3 hours 
More than 3 hours 



6 * Supply direction for difficult items. Two basic types of directions 
can be used:, those which force respondents to choose only one of the 



response categories; those which allow respondents to select all re- 
sponses that are appropriate. The type of directions you use depends 
.on the purpose of your question. Suppose that as a program admini- 
strator you are concerned with the sources of information the teachers 
use fc^jtevelop IEPs. Your question might read: 

> 

Identify the information you use to develop IEPs (check all that 
apply). 

standardized assessmeot results 

parent reports 

j observation 

curriculum assessment tools 
other, please specify 



s question indicates all the information that is used and the 

V, 



extent of use of each response. If you wanted to know which source 
was used most extensively, the directions might state^^Check the one 
source you most often use." Directions also ensure consistent answer- 
ing; procedures.. Without specific directions, some respondents may 
check all of the appropriate choices, and others may only check the 
mo$t appropriated The data then might be weighted or biased.. 

) Another selection item requires the respondent to rank (responses 
oriqijalitative statements. To rank a response, the respondent orders 
thp qualitative statements according to criteria *st|rited in the stem. 



If you are trying to ascertain parents' preferences for parent meet- 
i rigs dir activities, for example, a ranking item might look like this: 



Rank the following activities from 1 to 5: 5 being the one in 
which you most like to participate, 4 being the one you would next 
prefer, and so forth* Each activity should be ranked only once, 
and no number should be used more than one time • 



classroom visits/observations 
parent group meetings o 
IEP conferences 
Volunteer Training Program 
Other please specify) 



Data from ranked items can be analyzed to produce results very 
similar to results from qualitative items, since frequency of response 
and percentage are the basic types of analyses associated with all 
qualitative items. The data also can be used in the same way as demo- 
graphic data. For example, by dividing a group of respondents accord- 
ing to their response on item "age of child," you can determine if 
parents of younger children amswer certain-questions differently than 
parents of older children. 

.Supply items . This type of item differs radically from items previously 
discussed. In a supply item, the respondent must produce th^gsponse to 
an item rather than select from a set provided in the questionnaire. Sup- 
ply items are stems only. This type of item is particularly effective 

• • . ■ ■* . 

when the domaiff of responses is unknown, or when in-depth information 

: .' 

about a subject is sought, such as a- person's feelings, suggestions, or an 
explanation of events. The data gathered from these* types of items can 
render a rich description of happenings and can provide very detailed data 
because the respondent is not limited to general response categories of 
selection items. Basic disadvantages, are two: first, it is hard to get 
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responses to open-ended questions; second, aggregating the data is ex- 
tremely time-consuming* * 



1. Whenever possible, appropriate units should be specified for supply 
items. As noted previously ,.r€>spogdent-specific information is one of 
the most appropriate uses of the supply item. For example, you might 
be interested in the average number of hours spent studying per week, 
height and weight, hours spent working, monthly or weekly wages* etc. 
When you write items that elicit this type of information, tell re- 
spondents the unit of measurment with which they should answer. For 
^-example, if you are surveying parents to determine how much time they 
work with their children on project-related activities, you could ask 



the following open-ended question: 



(How much time do you spend working with your child on project- 
related activities? 



The question, as it stands, would allow all sorts of responses, 
including: "about 10 hours per week," ^pftiy on' the weekends and full 
time during the summer," "about 2 hours per day/ or "one week out of 
four." If you want answers comparable across respondents, sucji as 

of'tin 



relating the amount 



ime working to the developmental progress of 



children, it is essential t^at the item stem require more specific , 
answers. A better *i tern might be: * 

* During the time that your child was enrolled in the program, what 
was the average number of hours per week that you spent working at 
home with your child on project-related activities? 



hours per week 
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Another examp^ reveals a different aspect of the problem. In a 
recent survey of college administrators concerning volunteer activi- 
ties on their campus; a question was asked about the amount of money 
a I located at their institutions for volunteer activities. A perusal 
of these results revealed the fo l lowing responses: , 

: / » 

less than $0 ' 

^866 . 

$4,000 - 

525 

around $10,000 

* J 

How is it possible to reasonably aggregate these results, since 
the degree of accuracy varies so greatly? We assume that the person 
who said "$866" was accurate to the nearest dollar, though this pro- 
bably is not true ttf the person who replied "around $10,000," Some 
administrators answered to the nearest dollar, while others answered 
to the nearest thousand dollars* If the, purpose of this question is 
to aggregate data across institutions^ it is necessary to include not 
only t^ units (in this case dollars), but also several examples. The 
question should read: 



Please specify to the nearest one hundred do I l r ars the amount of 
money which is budgeted to your Volunteer program. For example: 
if your budget is between 0 and, $49, enter 0; if your budget is 
between $50 and $149, enter $100; if your budget exceeds $1,000, 
please report it to the nearest one hundred do 1 1 ars. 



Provide sufficient space for responses. Nothing is more, frustrating 
than to receive a questionnaire that does not leave enough space for 
your responses, in general, designers should cpasider the type of 
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responses they are looking for and should provide space accordingly. 
Remember, handwriting varies; if lines are provided, don't put them 
too close tqgether. If you expect respondents to write longer re- 
sponses to some questions than to others, allocate space accordingly. 



Before leaving the^disejjssion of the various item types and their 
criteria, one general statement should be made concerning their individual 
uses in a questionnaire. When designing any type of questionnaire for 
whoever purpose, select those item types which elicit the most reliable 
any valid responses concerning the variables addressed. Thw; means that 
for any instrument, do not limit the format to one type of ir^p; rather, 
vary the format as appropriate. " 



V 
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VII. STRUCTURE AND FORMAT 

• . -■ • 4} ■ „v 

After writing your items, directions, and demographic section, put the 
questionnaire together in a short, clear, final form/ Several criteria can be 

applied at this phase of the instrument's construction, 

* ■ 

A, Items should be grouped according to item types or similarity of content , 

f ^ * 

A logically structured questionnaire is easy to fill out and therefore 
helps increase the response rate. Your choice of format should depend on 
the purpose, the variables, and tjie item type selected. When long, lists 
of items are used,*lt is sometimes appropriate to set the groups of items 
apart as Subsections of the instrument, 1 . Once again, the purpose is to 
simplify the task of filling out the questionnaire,^ If you categorize 
item types into subsections, write directions for each subsection, 

B, Group items within sections accordin/^to ease with which they can be an -f 
s we red. If your instrument is made up of both selection and Supply type 
items, the selection type should precede* the supply types, since a greater 
number of respondents are likely to check rating scales than they are to 
fill in o^en-ended qu^ja^ions. This criteria helps ensure that respondents 
do not get discouraged by the more difficult items before they have an- 
swered the simpler ones, 

\ 

C, Length of the instrument should relate to respondents and purpose. In- - 
strument designers usually face a dilemma when they decide how long to 

^inake the instrument. They want to get as much information's possible, 
while at the same time obtaining the greatest response rate. Generally, 
short questionnaires get' higher^sponse rates than long, questionnaires. 
When you decide on the length, determine your respondents 1 tolerance level 



for answering questl'ons^and balance that level against the mi nimum amount 
of information you require. The kind of motivator provided for, the re- 
spondent also should affect the length of the instrument. Highly moti- 
, vated respondents will answer more questions than respondents who have no 
incentive to fill out the instrument. Finally, never ask questions whose 
answers you hatfe neither the^time nor ability to analyze. * 

D. The instrument should be clearly reproduced. ^Tha impact of a well-writterr 
questionnaire can be negated simply by typographic errors, ^r blurred or 
unclear copy. A respondent will not be inclined to take seriously a messy 
j/^^y questionnaire. ^Photocopies generally are clearer^than mimeographs, and 
the former make the instrument look more professional . Final ly, nothing * 
takes the place of a good proofreading. 



SUMMARY 

o 

When you develop or critique any instrument, refer to Figure 6 (next 
page) to see all thl criteria explained throughout this book. This list can 
be used as a guideline in the development process or as a critical tool by 
separately applying each of ^the criteria to your instrument, note that out- 
line numbers and letters in Figure 6 are keyed to appropriate discussions 
within the text of this book. 
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Figure 6. Guidelines arid Criteria for Constructing Questionnaires 



I, FOCUS 
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A. The specific purpose(s) of the questionnaire should be determined. 

B. Identify the persons who will use the results and identify their 
reasons for wanting them, 

C. Identify the major categories of information you seek, 

D. Identify the specific group(s) that will answer the' questionnaire. 

E. Identify the type of information you- seek, 

II. TITLE 

A. The title should reflect the content of the instrument. 

B. The* title should be concise and to the point. 

C. v The title should be written in plain English. 

III. INTRODUCTORY STATEMENT « 

A. The introductory 'statement should include a t^ief summary of the 
instrument's purpose. ■ ? . V 

B. The introductory statement should include >ah appropriate statement 
concerning the respondents 1 anonymity. 

C. The introductory statement should use a "motivator '' to encourage the 
respondent. , 

D. The introductory statement should be written at the language level 
of the respondents. 

IV. DIRECTIONS 

* A. Directions should be complete, unambiguous, and brief. 
.B. Directions should be writter^at the language level of the , 
respondents. J 

C. Directions should tell resppndents to whom they should give the 
instrument once they have completed it. 

D. When the questionnaire is accompanies by a separate answer sheet, 
the general directions should specify how the respondent should fill 
out the answer sheet., > 

E. The general directions should explain to the respondent how to deal 
with items which are not applicable. 

F. The general directions should specify the approximate amount of time 
required' to complete the instrument. 

V. DEMOGRAPHIC SECTION 

A. The demographic information should be limited to only those variables 
that will be used to answer specific questions. 

B. The demographic section should reveal the relationship of the 
respondent to the object of measurement^ w * 

C. Certain items in the demographic sections should be optional. 1 

D. Language in the demographic section should be appropriate to the 
language level of the respondents. 

E. Demographic items should be grouped together. 
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VI. WRITING ITEMS . 

A. Rating scale 

1. The stems of rating scale items should be written tridimen- 
sional ly. 

2. The response sets to rating scales should be written unidimen- 
sional ly. * 

3. The response set of a rating scale should be tied logically to 
the stem. 

4. The level of specificity of the stem should answer the user's 
needs. 

5. Use the rating^scale item only when a more direct method is not 
avai table. 

6. Directions for using the rating scale should be included, along 
with appropriate examples. 

a. Directions for a numerical rating scale 

b. 'Directions for a graphic rating scale 

c. Directions for a Likert scale 

7. The language used in rating scale items and responses should be 
written at the respondents' level of understanding. 

8. The type of rating scale format selected should be easily under- 
stood. . , o 

9. Rating scale items should not elicit biased responses. 

10. The response sets for a rating scale should present choices 
along similar degree continuums. 

11. Avoid using global terms in response sets. 
12; Rating scales should include from three to seven categories. 

•13. Choose the format for your rating scales that corresponds to the 

< sophistication of respondents. 
14. Allow for a "not applicable" response when appropriate. 
I 15. Choose an analysis strategy appropriate to the data;produced 
/ and the uses for which thejquestionnaire was designed. 

( B. Qualitative selection items 

1. State clearly and unambiguously the stem and responses. 

2. Use language appropriate to the respondents. 

3. State the stem unidimensiona I ly. 

4. Write an exhaustive response set. ta 

5. The response categories sjiould be mutually exclusive. 
^^6. 4 Supply directions^ for difficult items. 

-C. Supply items 

1. Whenever possible, appropriate units should be* specified for 
supply items. * • 

2. Provide sufficient space for responses. 

> 

VI I o STRUCTURE AND FORMAT 

A. Items should be grouped according to it^m types or similarity of 
content. 

B. Group items within sect iOTfs- according to ease with which they can be 
answered. * » 

C. Length of the instrument should relate to respondents and purpose. 

D. The instrument should* be clearly reproduced. « 

*'"-••■» 
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